SPECIAL REPORT ON FUTURE COMPUTING 



Is neural computing the key 
to artificial intelligence? 
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Kirk: You killed two of my crew! 

Nomad: Creator, your biological units are 
inefficient. 

irk: Nomad, it's time I told you who and* 
hat you are. I'm a biological unit and I 
created you! 

— Captain Kirk, talking with Nomad in the 
'The Changeling" 



Mike Donlin and Jeffrey Child 



When people speculate about whether com- 
puters can learn to think, evil robots, such 
as Nomad running amok on the Starship 
Enterprise, often enter the conversation. In the Star 
Trek episode, Captain Kirk talks the sinister Nomad 
into blowing itself up, but the notion of a computer that 
can control human destiny makes many folks nervous. 
Computer professionals scoff at such silliness, even 
while acknowledging that advances in hardware and 
software have given computers the ability to emulate 
some human traits. 

People have been fascinated with the idea of a ma- 
chine that could think since the days of eniac, one of the 
first "electronic brains." eniac had 30,000 vacuum tubes 
and 50,000 relays, filled a large room and could rip 
through mathematical calculations at a blistering 13 
operations a second. 

Presently, computer technology resides somewhere 
between eniac and Nomad, but advances in artificial 



intelligence, and particularly in neural networks, have 
caused a surge of interest in the thinking power of 
computers. 

I Nothing new about neural networks 

The concept of neural networks has been around in 
some form since World War II, but it's only in the last 
six or seven years that working products have been 
developed that attempt to "learn" about and predict 
reality. In their infancy, neural networks and neural 
computing were the work of theorists who observed 
similarities in the way that computers and humans 
think. In both cases, a large amount of information is 
manipulated by breaking it into small particles — using 
gates in computers and neurons in humans. Gates 
handle data by fluctuating between an "on" and "off' 
state, and neurons do the same by firing (on) or not 
firing (off). 

Scientists have postulated that human thought oc- 
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curs when two neurons fire simul- 
taneously — and that their connec- 
tion, called a synapse, is given more 
weight than would be the case if the 
neurons were connected but not fir- 
ing. Because of these similarities be- 
tween human and computer 
thought, researchers have begun to 
explore ways to embody the struc- 
tures of human intelligence in ma- 
chines. 

In the case of connecting neurons, 
either through hardware or soft- 
ware, to emulate the brain, the task 
has been daunting. The human 
brain contains 100 billion neurons, 
each connected to 10,000 others by 
synapses. Building such a complex 
computer is a ridiculous idea, even 
with the staggering advances made 
in computer technology in the last 
twenty years. 

Also, neural networks were dealt 
a blow in 1969 when Marvin Minski 
and Seymour Pappert wrote a book 
called Perceptrons, which postulated 
that neural network research was a 
waste of time. Minski, one of the 
founding fathers of the artificial in- 
telligence movement, refused to be- 
lieve that software could simulate 
the behavior of human neurons. 
Minski's vision of artificial intel- 
ligence (Al) was far more comprehen- 
sive than just neural network tech- 
nology, and he scoffed at those who 
Wanted to reduce his broad theories 
to a set of equa- 
tions that could 
solve only sim- 
ple problems. 
Many experts 
blame this book 
for derailing 
neural network 
research and en- 
couraging the 
expert-system 
theories favored 
by the authors. 

Expert sys- 
tems, in turn, 
have fallen out 
of favor in recent 
years, because 
they use a pro- 
hibitive amount 
of computing 
power to solve 
problems. Al- 
though there are 
some areas 
where encoding 
the skills of an 
expert and pro- 
gramming a 




"It's really important to 
understand that neural 
computing has nothing 
to do with building 
brains." 

— Casimir Klimasauskas, president of 
NeuralWare, Pittsburgh, pa 

III 

computer to carry them out seem 
feasible, for most complex tasks the 
intuition of an expert is simply too 
difficult to understand or too time- 
consuming to write out. 

"To make an expert system work, 
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In this simple neural network, a layer of inputs lists the various characteristics of 
fruit. When the network gets a stimulus from one or more of these inputs, it re- 
sponds with an output. If the network guesses incorrectly, it adjusts the correlation 
of the internal connections or synaptic weights until it gets a correct answer. 



you have to know why the expert 
makes decisions," says Steve Bis- 
sett, senior vice-president at Synap- 
tics (San Jose, ca), a neural network 
ic firm. "The problem is that most 
experts can't tell you all the rules 
that go into their decisions. They use 
past knowledge and intuition uncon- 
sciously, and that makes their 
knowledge difficult to codify. Even if 
you could write out enough knowl- 
edge to program into a computer, the 
amount of data would be so large 
that it would be prohibitively expen- 
sive to program and use." 

Expert systems are by no means 
finished, but there's also been a re- 
newed interest in neural computing 
in recent years, particularly in the 
fields of signal processing, forecast- 
ing and pattern recognition. Even 
though there are different ways to 
emulate the neural connection 
model of the brain, both in hardware 
and in software, all neural networks 
share certain common charac- 
teristics: they use artificial neurons 
that are connected to at least one 
other neuron, and they create their 
own representations of reality based 
on some form of learning model. 

I Learning by example 

Fundamentally, all neural networks 
learn by association. For example, a 
neural network can learn to identify 
an apple by associating the inputs 
"round," "red" 
and "fruit" with 
the output "ap- 
ple." The neu- 
rons in a neural 
network are 
usually orga- 
nized in three 
layers: input, 
hidden and out- 
put. Sometimes 
more than one 
hidden layer is 
used for complex 
analysis. 

There are 
many ways that 
neural networks 
can learn, but 
the most com- 
mon way is 
through exam- 
ple and repeti- 
tion, also called 
back-propagat- 
ion. Each time 
an input is given 
to the network 
("round," "red" 
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or "fruit" from our example), it gives 
an answer. Naturally, when a net- 
work is new, the guess will probably 
be wrong. But over time, as the net- 
work gathers more and more data, 
it will begin to zero in on the right 
answer. When it's fully trained, it 
can deliver an answer that's more or 
less accurate, depending on the com- 
plexity of the task. 



Every time the network guesses 
wrong, it adjusts its internal connec- 
tions until it gets the right answer. 
These adjustments are made via 
synaptic weights, which give rela- 
tive importance to data as it's ap- 
plied to a task at hand. These 
weights can be implemented in 
hardware or software, and it's this 
ability to gauge the importance of 



data that separates neural comput- 
ing from a purely digital comput- 
ational process. Although the syn- 
apses and weights can be made up 
of analog circuitry, digital compo- 
nents or software, the weighting 
procedure makes neural computing 
appear to have an analog nature — a 
characteristic that's especially im- 
portant in performing tasks such as 



Neural computing: What it is and what it isn't 



l^igital computers ushered in the in- 
formation age, and as they have be- 
come smaller, more personalized and in- 
creasingly powerful, they've touched 
more and more aspects of our lives. But 
as brains for robots, computers still 
leave a lot to be desired. Recognizing 
natural objects in the real world, for ex- 
ample, is well beyond the grasp of mod- 
ern technology. Are we just waiting for 
more computing power in a smaller 
package, or are we lacking a fundamen- 
tal ingredient? 

Computers perform logical computa- 
tion. They operate on precise input infor- 
mation with a programmed sequence 
of instructions and produce a precise 
output. Computers are much better 
than humans at operations such as long 
division, yet, when it comes to pattern 
recognition, even insects process infor- 
mation better than the most powerful 
computer. 

Scientists have been studying the 
brain for decades, trying to understand 
how biological computation works. 
Over the last ten years, a number of sig- 
nificant advances have been made and 
the embryonic field of neural networks 
has been born. 

An artificial neural network is a com- 
putational structure similar to its biological 
counterpart, yet much simpler, even when 
compared to a very small portion of the 
brain. Nonetheless, artificial neural net- 
works, simulated with digital computers, 
have already produced excellent results 
when applied to some real problems, 
such as predicting the outcome of horse 
races or playing backgammon. In many 
cases these results have been better than 
the best of the traditional logical or rule- 
based approaches. 

Artificial neural networks perform 
what might be called intuitive computa- 
tion. Rather than being programmed 
with a set of rules, they learn by exam- 
ple; they self-organize. A programmer 



isn't required to fig- 
ure out all the rules 
of the problem — a 
task that can be ex- 
tremely difficult — 
and then write a 
software program 
embodying the 
rules. Rather, neural 
networks discover 
the rules for them- 
selves through the 
process of training. 
A neural network 
contains a number 
of weighted parame- 
ters that are modified according to a 
learning rule, depending on the net- 
work's response to training examples. 

Much of the research into neural net- 
works has been performed using mod- 
els built in software, simulating a neural 
network with a digital computer. While 
invaluable for research, this approach 
has limitations in any practical applica- 
tion that requires quick results, because 
the computer calculates the effect of 
each interconnection (or synapse) one 
at a time, whereas nature does it all 
simultaneously. 

I Biology, analog or digital? 

Nature performs its computational func- 
tions using billions of neurons, each hav- 
ing thousands of synapses. To fully exploit 
the massive parallelism of the neural net- 
work structure, it's necessary to develop 
electrical circuits that approach nature's cir- 
cuits in density, speed and power dissipa- 
tion. Unfortunately, digital circuitry falls far 
short of the mark. 

Analog circuits, on the other hand, 
can perform complex computations by 
exploiting the physics of semiconduc- 
tors, which often mirrors the equations 
governing the behavior of biological 
neurons. In appropriate structures, this 
can lead to advantages of 100-to-1 in 




function density and 10,000-to-1 in 
power consumption per function, com- 
pared with digital circuits. 

The set of weights that embodies the 
learning of the network is stored in 
floating-gate structures; each analog 
weight value corresponds to an equiva- 
lent amount of electrical charge perma- 
nently stored in the floating gate. Synap- 
tics calls this technology adaptive 
analog vlsi. 

Despite their advantages, analog cir- 
cuits can't match the density and ener- 
getic efficiency of nature, but the perfor- 
mance of analog circuits offers such a 
dramatic leap beyond digital circuits — 
and the precision of the required com- 
putation is so low — that they open up a 
vast new realm of everyday applications. 

Synaptics solves pattern recognition 
and other problems by combining intu- 
itive computation with logical computa- 
tion. These two complementary forms 
of information processing are both es- 
sential in solving problems associated 
with autonomous-intelligent machines, 
that is, machines capable of naturally in- 
teracting with the real world. 

Dr. Frederico Faggin, co-founder, 
president and CEO, Synaptics, 
San Jose, CA 



COMPUTER DESIGN OCTOBER 1992 89 



SPECIAL REPORT ON FUTURE COMPUTING 



Neural net chip speeds check reading and verification 




You could find neural computing technology as close as your local gift shop. The 
Onyx check reader made by Verifone uses an analog-based neural network chip de- 
signed by Synaptics. With this chip, the unit can learn what a good check looks like, 
enabling it to handle a wider variety of checks — even checks that are folded, wrin- 
kled or badly printed. (Thanks to the Forget-Me-Not gift shop in Auburn, CA.) 



Have you ever stood in line at a 
bank, the supermarket or a gift shop, 
waiting to cash a check and wondering 
why it takes so long? Chances are, it 
has something to do with waiting for a 
check to clear. Or maybe it's a delay 
caused by the cashier's inability to read 
a check. 

To address this problem, Verifone 
(Redwood City, ca), a maker of transac- 
tion automation systems, asked itself 
how you outfit a check reader with 
enough computing power to handle 
any check, even those that are difficult 
to read. Checks in this category include 
those that are folded, wrinkled or im- 
properly printed. Worse, ink density of- 
ten varies from one set of characters to 
another. These variations can cause a 
great deal of trouble for check readers. 
Such machines either can't read the 
checks at all, or they read them inaccu- 
rately. Verifone wanted to make a check 
reader that could read 1 00 percent of 
the checks it sees. 

Enter Synaptics, a company that was 
developing an application-specific 
neural network chip. To make the check 
reader project feasible, designers at Syn- 
aptics reasoned that a simple neural net- 
work application would be best suited 
for its chip. Because there are a limited 
number of characters on a check, 
through 9 plus four special symbols, the 
Verifone application seemed an ideal 
candidate for a neural network. 

Working with Verifone, Synaptics de- 
veloped an analog neural network chip, 
the 1-1000, specifically for reading 
checks. The design is historic in that it's 
the first commercial application using a 
neural network chip. In fact, the chip is 
so application-specific that it has a built- 
in lens through which it reads the im- 
ages. The lens forms an infrared image 
on a "retina." The chip then tries to de- 
code the character from this image. 
Making a number of computations, the 
chip comes up with an answer. It may 
see a "9," for example, assigning an 85 
or a 95 percent probability to its answer, 
depending on how good the image is. 
A perfectly printed character with per- 
fect ink density could receive a 99.5 per- 
cent probability. To allow for checks that 
are printed badly or are wrinkled or 
folded, Verifone trained the neural net- 
work to pass anything with a probability 
of over 80 percent. 



In an independent laboratory test, the 
neural-based check reader, dubbed the 
Onyx, was accurate 99.6 percent of the 
time. More important, it was capable of 
reading every check going through it. 

I Software and hardware support 

While Synaptics was primarily responsi- 
ble for the chip design, Verifone devel- 
oped software and hardware to support 
the neural net chip. A 68HC 1 1 , Mo- 
torola's 8-bit microcontroller, controls all 
the processes in the machine and inter- 
faces between the 1-1000 and the soft- 
ware. While the 1-1000 does the decod- 
ing, the Onyx also has neural network 
software. The job of this software is to 
make sense of what the chip tells it. 

If the chip tells it, for example, that 
there's an 80-percent probability it's 
reading an "8," the software tries to de- 
termine whether the result makes 
sense. It captures frames to monitor tim- 
ing as the check goes through the 
reader. Each frame runs for a different 
timing interval. The software determines 
if the acceleration of the frame makes 



sense; it compares its conclusion to the 
result produced by the hardware. The fi- 
nal result is based on this comparison. 

For memory, the Onyx has 28 kbytes 
of sram. Battery-backed sram is used in- 
stead of rom to permit easy updates. If 
new types of checks are printed, 
Verifone can simply update (by retrain- 
ing) the software in the company's lab. 
The software can then be downloaded 
to check readers over the phone lines. 

Synaptics' neural net chip does more 
than just read the numbers, however. 
Using the magnetic properties of the ink 
used to print the bank number, the 
neural net can also determine whether 
or not the check is counterfeit. The ink 
used has a high iron-oxide content, and 
since most counterfeit checks are pro- 
duced using a color copier, they 
wouldn't show any iron-oxide content. 
The infrared image indicates iron oxide 
in the ink based on the frequency of the 
emission received. If there isn't any iron 
oxide, no image is recorded, and the 
check is rejected. 
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pattern recognition or financial fore- 
casting. 

"There's a key difference between 
neural computing and digital com- 
puting," Synaptics' Bissett points 
out. "Traditional digital computing 
is like the left-brain or logical think- 
ing that we do. The computer re- 
ceives a set of rules or programs, 
then takes input and produces out- 
put based on those rules. By defini- 
tion, it's only as good as the rules 
that guide it. Neural computing is 
more like right-brain thinking, 
which is intuitive. If you 
wanted a computer to read 
handwriting, you could try 
to write rules that would 
make it recognize an 'S,' for 
example, and that would be 
a very linear, but increas- 
ingly complex, way to solve 
the problem. That's not the 
way we do it. Our circuits 
are connected in parallel, 
and compare a shape to pre- 
vious knowledge to try and 
categorize it. An important 
distinction, then, is to try 
and teach by example 
rather than programming 
by rules." 

The correlation of neural 
networks to the way our 
brains work is what makes 
them suited to applications 
that need experiential 
learning, but is neural com- 
puting really thinking? In a 
word, no. Neural networks 
are patterned after the ar- 
chitecture of the brain, but 
in reality their ability to 
think is far more primitive 
than that of a common 
housefly. As a matter of 
fact, some experts scoff at 
the notion that neural net- 
works are related to human 
thought at all, other than in 
a purely analogous way. 

"It's really important to 
understand that neural computing 
has nothing to do with building 
brains," says Casimir Kli- 
masauskas, president of Neural- 
Ware (Pittsburgh, pa). "Neural net- 
works are a collection of 
mathematical techniques that let 
you fit formulas to data, curves to 
data, and group types of data to- 
gether. Neural networks could have 
been invented by statisticians, phys- 
icists or mathematicians, but the 
people who invented them were cog- 
nitive psychologists and neurobiolo- 



gists, and so we ended up with the 
term neural networks. They have 
nothing to do with brains. I've found 
that if you try to explain neural net- 
works from a human-thought per- 
spective, people keep trying to fit 
them into a brain model, and it only 
confuses them." 

I Enter fuzzy logic 

In spite of such caveats, most people 
will probably continue to associate 
neural networks with human 
thought, particularly because much 




"Neural networks have a lot of potential in optical charac- 
ter recognition," says Dan Hammerstrom (background), 
founder and chief technical officer of Adaptive Solutions, 
"especially where you have mixed fonts, uneven spacing 
and handwritten letters. In our OCR system, we point the 
camera at a page and isolate the characters from surround- 
ing spaces and graphics. Our system then makes the charac 
ters uniform and passes them through a classification 
phase, where the letters are differentiated from one an- 
other and turned into ASCII code that the computer can 
use." 



of the learning process in a neural 
network takes place in hidden lay- 
ers or neurons, a processing para- 
digm similar to human thought. 
Some researchers are trying to un- 
ravel these hidden functions by us- 
ing fuzzy logic techniques to better 
understand, or even work in con- 
junction with, neural networks. Fu- 
jitsu (Kawasaki, Japan) is working 
on a system, for example, that cre- 
ates a fuzzy rule set based on "if- 
then" questionnaires that have been 
filled out by experts. These fuzzy 



systems are converted into a neural 
network, which learns about the 
task and refines its knowledge. Once 
the neural network has achieved an 
acceptable degree of accuracy, it's 
translated back into a fuzzy system 
so its operation can be analyzed. 
According to Fujitsu researchers, 
this model reveals the hidden varia- 
bles that develop in a neural net- 
work. By keeping the fuzzy rule sets 
and the neural networks as separate 
systems, the Fujitsu scientists be- 
lieve they can learn more about how 
each system works. 

But not everyone in- 
volved in combined 
fuzzy/neural computing 
wants to keep the discip- 
lines separate. There's con- 
siderable activity in the ai 
community aimed at com- 
bining the neural network's 
ability to create relation- 
ships with fuzzy logic's ca- 
pability to produce input 
and output information that 
spans a range of behavior. 
One such approach builds 
fuzzy operations into the 
learning techniques used by 
a fuzzy/neural controller. 
The resulting neural net- 
work learns to emulate a 
fuzzy controller, but with 
rules that can be altered by 
neural learning techniques. 

In spite of the promise 
that the combination of 
fuzzy logic and neural net- 
works holds, most of the 
work in this area is at the 
theoretical stage, although 
some practical applications 
have been demonstrated in 
medical imaging and flight 
simulation. To many 
theorists, however, the mar- 
riage of these two "smart" 
technologies seems inevi- 
table. 

Theories aside, most of 
the tangible products using either 
fuzzy logic or neural networks have 
come from research that treats each 
of these disciplines as separate enti- 
ties. In the field of neural networks, 
this means three categories of prod- 
ucts: neural network ICS, whose ar- 
chitectures are specifically designed 
for neural computing; neural net- 
work software, which uses standard 
microprocessors and digital signal 
processors to emulate neural net- 
work behavior; and neural network 
systems, which are turnkey neural 
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Analog & digital approaches 
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The 64 analog inputs to the 
etann device from Intel (top 
left) are capacitively stored 
for a limited time by taking 
the hold pin high. The 
ETANN's 10.240 synapses 
store weights as analog 
transconductance values, 
each producing an analog 
output current from an 
analog input voltage and a 
stored weight voltage. Cur- 
rents generated by each of 
160 synapses along a den- 
drite, or ± column pair, are 
summed to form the net in- 
put to a neuron body. The 
dendrite's sum of currents is 
then converted to a voltage 
and passed through a sig- 
moid function with voltage- 
controlled gain. The 
CNAPS-1064 (bottom left) 
from Adaptive Solutions is 
an array of 64 processing 
nodes (PNs), each with 4 
kbytes of on-chip SRAM. Each 
pn resembles a DSP and con- 
nects to three global buses: 
IN Bus, the data input bus; 
pncmd, the command bus 
that dictates the operations 
of the pn each clock cycle; 
and the OUT Bus, an output 
bus. 
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network computers. 

The first commercially 
available neural network- 
specific chip was the 
80170NX electrically trainable 
analog neural network (etann) de- 
vice from Intel (Santa Clara, CA). 
Introduced in 1989, the chip is a 
64-neuron, 10,240-synapse IC with 
inputs organized as two groups, ex- 
ternal and recurrent (or feedback); 
each input contains 80 x 64 synapse 
arrays. 

Intel has also released a develop- 
ment system so that you can simu- 
late, train and operate a high-speed 
neural network. Dubbed the Intel 
Neural Network Training System 
(iNNTS), the package provides two 
80170NX devices, two learning sim- 
ulation software programs, diagnos- 
tic software, a programmer inter- 
face, an adapter that can run on 
PC/AT-compatible computers, pro- 
gramming specifications, and full 
documentation. The iNNTS contains 
two learning simulation software 

continued on page 97 
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continued from page 92 

programs, iBrainmaker and Dyna- 
Mind. The iBrainmaker program, 
developed by California Scientific 
Software (Grass Valley, CA), lets you 
simulate the network learning pro- 
cess through back-propagation tech- 
niques. In back propagation, you 
present the network model with a 
data set representing the applica- 
tion problem. Through simulation, 
iBrainmaker then trains the net- 
work to produce a desired response 
to specific inputs by assigning 
weights to each of the chip's analog 
storage elements. Once the network 
has been trained to solve the appli- 
cation problem, the weights are 
downloaded or programmed into the 
etann device. 

The DynaMind simulation soft- 
ware, developed by NeuroDynamX 
(South Pasadena, CA), lets you sim- 
ulate back-propagation learning, 
but also performs chip-in-loop learn- 
ing. This technique optimizes the 
performance of the network by re- 
placing the software simulation of 
the chip's performance charac- 
teristics and specifications with an 
actual device. The neural network 
can then "learn around" any minor 
processing variations occurring in 
individual etann chips. 

Both iBrainmaker and DynaMind 
can also be used independently of 
the Intel development system to cre- 
ate neural network applications. 

I Fastest-learning chip 

A more recent neural network IC is 
the RN-200, a 256-synapse (16 syn- 
apses x 16 neurons) device that 
Ricoh (Tokyo, Japan) claims is the 
world's fastest-learning chip of its 
kind. The device boasts a front-end 
process of three billion connections 
per second and a learning speed of 
1.5 billion connection updates per 
second (cups) when running at 12 
MHz. In Tokyo, Ricoh demonstrated 
a desktop neural computer system 
that requires no software. Based on 
the first-generation RN-100 chip (a 
one-neuron device with eight syn- 
apses), the neurocomputer has a 
processing speed as high as 128 mil- 
lion neuron connections per se- 
cond — a figure that Ricoh says will 
increase when the system incor- 
porates the new RN-200. 

One of the first neural network 
ASICs comes from Neural Semicon- 
ductor (Carlsbad, CA). Its CNU3232 
has 32 inputs, 1,024 synaptic 
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ability to learn by association shows the technology to be promising for every- 
thing from fruit grading to target recognition for weapons systems. 

The apple grading system captures images and feeds them to a neural net- 
work, which eventually learns how to determine which characteristics affect 
an apple's quality. Once the network is trained, the system can classify an ap- 
ple by comparing its traits to the network's learned base. 

"The tank recognition system," says Ted Crooks, director of customer ser- 
vices at HNC, "was actually an experiment to prove that neural networks 
could be used in defense systems. Critics of neural networks cited an applica- 
tion where a trained neural system picked out tanks flawlessly until it was dis- 
covered that all pictures of tanks were taken in bright light and those without 
tanks were taken in shade or at night. As soon as those clues were taken 
away, the network failed. We proved that a neural network could indeed be 
trained to differentiate a tank's shape from other objects, and that experiment 
led to a real application." 

In this picture, a filtered image of two tanks is outlined so that the network 
can learn to differentiate them from other objects. 
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Neural computing challenges the status quo 



I or this Special Report on Future 
Computing, Computer Design inter- 
viewed Carver Mead, an expert on the 
subject of neural computing. Professor 
Mead is the Gordon and Betty Moore 
Professor of Computer Science at the 
California Institute of Technology, where 
he has taught for 20 years. He's also a 
co-founder and chairman of Synaptics, 
a company that develops neural net- 
work technology. Mead has pioneered 
in many areas of electronics, from the in- 
vention of the mesfet to silicon com- 
pilers and, recently, vlsi analog neural 
systems. 

Computer Design: After many years 
of use in academic circles, neural comput- 
ing now appears 
poised to move 
into practical com- 
mercial embedded 
applications.What's 
taken so long for 
this to happen? 

Mead: Your 
question reminds 
me of how people 
used to talk about 
parallel architec- 
tures in computers 
years ago. People 
said: "We do 
things in a top- 
down way. " And 
I'd ask: "Do you re- 
ally know what the 
'top' is?" I'd get 

these blank looks from people. But to- 
day, 1 5 years later, people are still sort- 
ing out what the top is in parallel com- 
puting. "Top-down" assumes you know 
everything in the beginning. That's not 
a very realistic view of the world. You 
don't know what's what in the begin- 
ning, so you have to evolve your under- 
standing along with the application. 

In the neural network business that 
top-down approach has translated into 
some rather abortive attempts to make 
general-purpose neural network chips. 
Those chips haven't worked well be- 
cause no one knows what architecture 
is right for any real application. There 
are a lot of interesting simulations done 
in research circles, but none of those 
are applications that have to work in 
real time, under real-world conditions. 

So then, rather than trying to general- 
ize from basically no information — 




which is what we're faced with today — 
it makes more sense to do specific appli- 
cations. Until you've done that, there's 
really no royal road to the top to see 
what is the general case. General cases 
don't come that way. Those come hard- 
fought after years of working with spe- 
cific cases. That's why I think its impor- 
tant for designers to solve real problems 
all the way out. Then we'll begin to ac- 
cumulate data that we can generalize 
from. 

CD: How will future advances in vlsi 
process technology influence the capa- 
bilities of neural network chips? Do you 
see any potential roadblocks? 

Mead: Silicon process technology is 
very relevant to neural computing. A lot 
of people have tried to 
invent brand-new tech- 
nologies to do neural 
networks. But it's im- 
portant to remember 
that we are riding on 
the coattails of a sili- 
con technology that's 
highly evolved. 
Hundreds of billions of 
dollars have gone into 
this most advanced 
technology that civiliza- 
tion has ever known. 
And to think that 
you're going to start 
from scratch with 
some other technology 
and do as well is pretty 
silly. At Synaptics 
we've been developing an adaptive 
analog technology that takes advantage 
of all the capabilities of process technol- 
ogy. As process technology evolves, it's 
immediately applicable to this adaptive 
analog approach. That's not true of 
other approaches to neural networks. 

I Transistors as analog devices 

CD: As I understand it, your neural 
net chip uses the transistors in digital 
semiconductors in an analog way. 

Mead: Yes. Transistors are analog de- 
vices. Let them be what they are. Digi- 
tal ic designers have had to work so 
hard to turn them into 1s and Os that 
they lose all that beautiful analog capa- 
bility. But if you let them exhibit that ca- 
pability, then you get to save a factor of 
1 0,000 in power consumption, as we 
did in our chip. It's really remarkable. 

The inputs and the intermediate sig- 



nals are inherently analog signals — 
they're typically faked out by binary 
numbers right now in a computer simu- 
lation, but that's not the effective way 
to use them. The effective way is for 
them to evolve in real time as analog sig- 
nals. Digital computers not only turn sig- 
nals into digital values, but they also use 
discrete time. Those discrete time 
stamps actually destroy information by 
aliasing. And because a neural network 
is nonlinear, there's no theory that tells 
you how much information you've lost. 
Using transistors in the continuous (or 
analog) domain gets rid of those prob- 
lems. We get back so much for using 
transistors in an analog way. 

Fortunately, the transistors in semicon- 
ductors never did know that they were 
supposed to be digital. They're inher- 
ently analog by nature. That means you 
can use them that way, and the only 
thing you have to be careful of is that 
not all the transistors are exactly the 
same. That's why the adaptive part is so 
important. Because, not only do you let 
the neural networks learn from their en- 
vironment, but they also adapt to 
changes in the environment. And one 
of the important changes in the environ- 
ment is the difference among the tran- 
sistors. If you do it right, the same para- 
digm that allows you to adapt to the 
outside world lets you adapt to differ- 
ences among the transistors that are on 
the chip. That lets you use a commercial 
fab technology to build sophisticated 
neural net analog processors. 

So far, we haven't seen any techno- 
logical roadblocks in this path. In fact, 
we're seeing that as the technologies 
are evolving for digital use, they're actu- 
ally evolving capabilities that we use in 
an analog way to make them even 
more effective. 

I The analog-digital continuum 

CD: It seems clear that neural comput- 
ing is not destined to replace traditional 
logical computing, by any means. It may 
even open up more opportunities for 
logical computing. How do you see the 
situation? 

Mead: The real world tends to pres- 
ent you with continuous values: the in- 
tensity of a pixel or the value of a 
waveform, to take two examples. And 
the digital world deals with discrete sym- 
bols: the letters of the alphabet, for ex- 
ample. At some point, the continuous 
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stuff gets translated into discrete sym- 
bols. In the most general terms, this 
translation process is called classification 

When I started in electronics, almost 
the whole system was analog. And the 
classifier was a relay. Then you had a 
contact closure and that was your dig- 
ital output. It turned on a light or a 
heater element in a furnace or some- 
thing. Now we've gone hard over the 
other way to where we now have, in to- 
day's world, a little bit of antialias filter- 
ing and a multiplexer on the analog 
side. Then there's an analog-to-digital 
converter, which you could think of as 
the most brain-dead classifier you can 
imagine. Because all it does is take 
analog values and convert them into bi- 
nary numbers which represent voltage 
changes. The computer is expected to 
manipulate these values as if they were 
numbers and eventually simulate a clas- 
sification scheme that outputs discrete 
symbols. 

The trend now is toward a much 
more balanced view of that picture. The 
analog preprocessing 
gets the data into a 
form where it's readily 
classifiable into appro- 
priate higher-level sym- 
bols. If you were doing 
speech, for example, 
you'd want to classify 
the data into 
phonemes. Our 1-1000 
chip classifies images 
directly into character 
codes. A character 
code is a lot more 
meaningful to the 
computer than the 
analog values of the 
pixels. Computers are 
the way to handle dis- 
crete symbols, but 
they ought to be appropriate discrete 
symbols. 

So we're seeing a return to letting the 
different technologies do what they're 
good at. You use an adaptive analog sys- 
tem to do all the preprocessing. You use 
an analog classifier to decide what the 
best classification is. That has a digital 
output which goes into a digital system. 

Many of the neural network chips 
available today are aimed at that classi- 
fier job. But they're leaving out all the 
analog preprocessing that makes the 
data fit to be classified. 



So, I believe that what we're headed 
for as a field is getting the picture to be 
more balanced, so the stuff at the 
analog end of the spectrum is done by 
adaptive analog technology and the 
stuff on the digital end is done by dig- 
ital computers (as it is today). And the 
classification in the middle, between the 
two, is done at a level which makes 
sense for the problem. 

I The classification problem 

CD: I guess you can shoot yourself in 
the foot if you make the classification 
problem too complex? 

Mead: If you make that too big a prob- 
lem, then it simply fails a lot. So you want 
to do those things at an appropriate level. 
And that's just plain good engineering. So 
we're not talking about replacing digital 
computers. It's just a matter of recogniz- 
ing which technology is most natural for 
each situation. 

CD: Today a lot of neural networks 
are simulated in software on large, pow- 
erful computers. 

Mead: Right. It 
takes too much 
digital computing 
for the size of the 
problem. That lim- 
its the applications 
to the very few, 
where time isn't 
critical and you've 
got the processing 
power of a Cray 
computer around. 
It's only by getting 
this balance in the 
technologies that 
we're going to see 
very widespread 
use of the neural 
paradigm for real 
systems. 

CD: Where do you see neural comput- 
ing five or 1 years from now? 

Mead: If you look at the continuum 
between analog and digital, we've gone 
all the way from 90 percent of the sys- 
tem being on the analog side to 90 per- 
cent on the digital side. We're headed 
back toward a balance of about half 
and half. Over the next few years, you'll 
see the analog side growing for applica- 
tions which have to interface to the real 
world. There's no doubt in my mind 
that's where neural computing and com- 
puting in general are headed. 




weights and 32 nodes supporting its 
activation functions. The one-byte 
digital inputs and outputs and the 
weight-storage SRAM are all accessed 
through an 8-bit I/O bus. Unlike the 
Intel chip, which uses analog ele- 
ments, the CNU3232 is a purely dig- 
ital device that's targeted at embed- 
ded system applications. 

"We refer to ourselves as a neural 
ASIC company," says Robert Bagby, 
president of Neural Semiconductor, 
"because we expect our customers to 
build neural network ICS of various 
sizes, topologies, precisions, and ac- 
tivation functions using our basic 
architecture. Neural networks are 
really multiple layers of nonlinear 
matrix multipliers. We build dis- 
crete circuitry for each and every 
neural multiplier, and place SRAM ad- 
jacent to that multiplier to store 
neural weight values. We also have 
a neural summation function built 
into the chip, so you have fully par- 
allel neurons and fully parallel syn- 
apses or weights. Because our archi- 
tecture is purely digital, designs 
based on it can be manufactured 
with standard processes for low-cost, 
high-volume implementations." 

The first neural network ic to find 
its way into a commercially availa- 
ble product comes from Synaptics. 
The chip, designed for optical char- 
acter recognition (ocr), hosts an 
analog sensing array, two neural 
networks and a digital controller on 
a single device. The chip is at the 
heart of a check reader being sold by 
Verifone (Redwood City, CA). "Until 
now, if you wanted to perform high- 
speed OCR, you were limited by the 
bandwidth between the sensor and 
the rest of the circuitry," says Syn- 
aptics' Bisset. "For most applica- 
tions using a TV camera, that rate 
was just 30 images per second. By 
putting the sensor on the same chip 
with the classification circuitry, we 
can do the same task thousands of 
times per second." 

I Other solutions 

Not all hardware solutions for 
neural network applications are 
based on neural-specific silicon. 
There are systems that use standard 
digital components for neural com- 
puting, as well as for non-neural ap- 
plications such as Fourier or Gabor 
transforms. The CNAPS-1064 from 
Adaptive Solutions (Beaverton, or) is 
such a hybrid — an array of 64 pro- 
cessing nodes (PNs), each with its 
own 4 kbytes of on-chip SRAM. A pn 
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DSP and neural nets team up for medical research 



Lny biomedical technician 
can probably do a better job 
than a computer of visually iden 
tifying reactions in blood cells. 
But when researchers at the Uni 
versity of California (Davis, ca) 
needed to classify 72,000 blood 
cell reactions each day, and do 
so accurately and consistently, 
they moved to an automated 
approach that combines digital 
signal processing (dsp) with a 
neural network simulated in 
software. 

At first glance, the applica- 
tion seemed like a simple image 
classification problem. Blood- 
cell images are captured using a 
high-resolution ccd (charge- 
coupled device) camera at- 
tached to a microscope. Each 
image is about a millimeter and 
a half in diameter. That area 
contains about 100,000 red 
blood cells. To induce a reac- 
tion, antibodies are suspended 
with the blood cells. After a pe- 
riod of incubation, a reaction 
may or may not occur. The pho- 
tos show four degrees of reac- 
tion into which the neural net- 
work classifies images. 

There's added complexity, 
however, because no two reac- 
tions are exactly the same. The 
network also has to account for 
rotated views of the exact same 
image. For example, if a human were 
looking at the image and felt he or she 
could better identify the reaction from 
another angle, the image could simply 
be rotated. For the computer this would 
be very complicated. You'd have to 
show the network how the image 
looked rotated 5 degrees, and that 
means the network would have to ana- 
lyze 72 different versions of the same 
image. In back-propagation neural nets, 
the difficulty of training increases pro- 
portionally to the number of training ex- 
amples raised to the third power. 

I Preprocessing needed 

"In neural networks, the more work you 
can do before you hand off the project 
to the neural net, the better off you 
are," explains Wasyl Malyj, associate de- 
velopment engineer at uc Davis. With 
this in mind a preprocessing step was in- 
cluded to extract from the blood-cell im- 




The neural network classifies blood-cell reactions into 
one of four types. The image in upper lefthand corner 
shows "no reaction" or a class reaction. The image to 
the upper right shows some traces of clumping indicat- 
ing a "weak reaction," or class 1. The lower left image 
shows even more clumping, indicating a "definite reac- 
tion," class 2. And finally, to the lower right the image 
indicates a "complete reaction, " class 3, has occurred. Be- 
cause these images are so complex (over a quarter of a 
million pixels), it's important to extract only the most rel- 
evant pixel data using DSP techniques. Otherwise, the 
neural network would require an unrealistic amount of 
computing power. 



age only its most critical data. 

To accomplish this, a two-dimensional 
Fast Fourier Transform (fft) is performed 
on the image's pixel data, converting it 
into the frequency domain. This pro- 
duces a compact feature vector. Sam- 
pling algorithms are applied to these 
vectors to extract useful information. 
The goal of the preprocessing is to take 
a very complex image with upwards of 
512 x 512 (or over 1/4-million) pixels 
and extract from that a few hundred 
bytes of data. "The preprocessing re- 
duced the amount of stuff that the 
neural net didn't need to learn, simplify- 
ing its structure, its training, and mak- 
ing it possible to implement the neural 
net with today's technology, " says Malyj. 

After the image is compressed into a 
complex feature vector, the vector is fed 
into the neural net. Cycling this informa- 
tion through the net lets it adjust its con- 
nection strengths, and in this way it 



"learns" to associate particular 
spectral patterns with particular 
reactions. 

Because of the preprocessing, 
the input stage of the neural net 
is typically 128 neurons. To imple- 
ment the network, the uc Davis 
researchers developed in software 
a custom-written back-propaga- 
tion simulator capable of building 
nets with three or four layers. The 
code was written to run on a 486 
working in conjunction with a 
Motorola 96002 floating-point 
digital signal processor (dsp). 

The input layer typically has 
128 neurons. The hidden, or mid- 
dle, layers usually have between 
12 and 64 neurons, while the out- 
put layer consists of 8 to 20 neu- 
rons. Four of the output neurons 
represent the discrete reaction 
classes, an additional output pro- 
vides a confidence metric and the 
remaining outputs code for a vari- 
ety of possible error conditions, 
such as cracks in the plastic tray, 
bubbles in the sample, shadows 
cast by bubbles in the mineral oil 
covering the specimen, lack of 
blood or reagent in the reaction 
well, and proper focus. 

I Training the net 

According to Malyj, it took only 
about six hours to train the neural 
net. The dsp hardware helped 
boost its speed. A training set of about 
800 images was shown to the neural 
net, along with the classification under 
which each image belongs. Once the 
neural net was trained, technicians be- 
gan to feed it images that it had never 
seen before for classification. 

The network can be adjusted for vari- 
ous performance levels. "If we tell the 
neural net to classify absolutely every- 
thing, it's accurate to percentages from 
the high 80s to the low 90s," says 
Malyj. "That's not as good as a human. 
But if we tell the net to classify only 
those images about which it's 'confi- 
dent,' and to flag the others for us to 
take a look at, then it classifies about 
85 percent of the images at better than 
99 percent accuracy." 

Researchers can then take the remain- 
ing 1 5 percent of the images and, after 
they've been scored by a human, use 
them to retrain the network. 
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resembles a simple digital signal 
processor (dsp), and can be pro- 
grammed for a variety of applica- 
tions. At 25 MHz, the device can 
compute 3.2 billion multiply-adds 
per second. The chip falls some- 
where between PC-based software 
solutions for neural network appli- 
cations and silicon 
that's targeted at those 
applications. 

"Given the perfor- 
mance of today's PCs and 
workstations, you can 
emulate a lot of neural 
network applications in 
software alone," says 
Dan Hammerstrom, 
founder and chief techni- 
cal officer at Adaptive So- 
lutions. "But from a sili- 
con standpoint, there are 
two approaches you can 
take. One is a device like 
ours, which can be cus- 
tomized for a certain do- 
main of applications but 
is flexible enough to be 
programmable. The 
other is a chip like Syn- 
aptics', which is designed 
for a specific task and 
which may have a 
price/performance advan- 
tage for that application. 
Both have their place. For 
complex tasks such as Kanji charac- 
ter recognition, it's simply not feasi- 
ble to build a custom chip. That's 
where flexibility pays off." 

I Software-only solutions 

The remaining category of neural 
network-specific products is soft- 
ware-only solutions, which emulate 
neural networks on workstations, 
PCs and mainframes. These pro- 
grams use either standard micro- 
processors or dsp chips to perform 
neural networking tasks, and are 
available from dozens of companies. 
The applications are as diverse as 
the companies that offer them, but 
most products are being used for 
pattern recognition, financial analy- 
sis and defense-related projects. 

In essence, most neural network 
tasks are based on some form of 
pattern recognition. Sometimes the 
sought-after pattern is a visual 
one — for example, a system that 
sorts good fruit from bad on a con- 
veyor belt. In this application, the 
network must be trained to look for 
a vague trait such as "quality." "The 
customer who wanted to grade fruit 



had to train the network to recog- 
nize color patterns that distin- 
guished good apples from bad," says 
Ted Crooks, director of customer ser- 
vices at hnc (San Diego, ca), a 
neural network software vendor. 
"By repeatedly presenting data to 
the network, he trained it to discern 




cell structure, some physicians are 
using neural networks to help them 
with diagnosis and prognosis. "One 
of our customers is a neurosurgeon 
who's using neural networks to pre- 
dict potential IQ loss after brain sur- 
gery," says Jim Blodgett, director of 
marketing for California Scientific 
Software. "Traditional 
statistical methods 
have been used in the 
past, but they lose pre- 
cision in the middle of 
the bell curve. At each 
end, either with slight 
damage or severe dam- 
age, the statistics are 
fine. But in the middle 
of the curve there can be 
large variations. Neural 
networks can use fac- 
tors such as the severity 
of an injury or a pa- 
tient's medical history 
to make predictions of 
an operation's out- 
come." 



"Not all neural networks are created equal," says Steve Ward, pres 
ident of Ward Systems Group. "A traditional feed-forward net- 
work produces one and only one set of outputs from a given set 
of inputs after it's trained. A recurrent network, on the other 
hand, may produce many different sets of outputs from one given 
input set, depending on when in time the input set is presented to 
the network. Recurrent nets not only learn individual input pat- 
terns, but learn sequences in those input patterns as well." 



the important relationships that de- 
fine good quality — for example, 'this 
is premium,' 'this is grade six,' 'this 
is rotten,' and so on. He didn't define 
what made up these classifications; 
he just gave examples of what char- 
acteristics are needed to place a 
piece of fruit there." 

Visual pattern recognition is also 
being applied to medical research. 
Some hospitals are using a neural 
network to sift through hundreds of 
slides to detect anomalies in blood 
cells or tissue samples. Early results 
from these applications show that 
neural networks achieve a surpris- 
ing level of accuracy when they're 
compared to a human performing 
the same task. As with most appli- 
cations where a computer equals or 
bests a human being at a task, fa- 
tigue is the deciding factor. Although 
human expertise may hold the up- 
per hand in picking a cancerous cell 
out of thousands of normal cells, hu- 
man fatigue can cancel out some of 
that expertise, and the capabilities 
of human versus neural network be- 
gin to equalize. 

In addition to recognizing flaws in 



I Financial uses 

While neural network 
research in medicine 
makes for dramatic 
reading, there are other 
applications, particu- 
larly in the realm of fi- 
nance, that might have even greater 
ramifications. Banks are relying on 
neural networks to do everything 
from predicting loan eligibility to 
spotting credit-card fraud. In the 
case of loan eligibility, the network 
is taught to examine the factors that 
would make a good loan applicant, 
based on profiles of good and bad 
loan recipients. The usual criteria 
are included in the data, such as 
income, time on the job, credit his- 
tory, and so on, but neural networks 
look for unusual patterns or rela- 
tionships which might escape a hu- 
man, particularly a human who 
looks at dozens of applications a day. 
In the case of credit-card fraud, the 
network might look for spending 
patterns which are unusual, such as 
someone who purchases a wide- 
screen TV, a trip to Europe and a 
gourmet meal in a 24-hour period. 

The common denominator in 
these applications is training the 
network to look for subtle shifts in 
patterns which are crucial to mak- 
ing a final decision. In the case of 
stock market predictions, one ana- 
lyst said that a day on Wall Street is 
akin to being hit with a dozen data 
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firehoses at once. The trick is to 
ascertain which droplets of data in 
that enormous stream are the ones 
that will affect the market. 

"Naturally, the key here is giving 
the system data," says Steve Ward, 
president and technical director of 
Ward Systems (Frederick, md). "One 
of our customers, an investment ad- 
visory firm, is using our NeuroShell 
software to create a Standard and 
Poors stock market prediction sys- 
tem. The client uses technical indi- 
cators to predict short-term 
changes, and a combination of tech- 
nical and fundamental indicators to 
predict long-term changes. The da- 
tabase includes 12,000 companies 



and looks for fundamental data pat- 
terns in areas such as growth and 
cash flow yield, as well as cross-sec- 
tional analysis or valuation change 
to compute expected return. Early 
experiments using a model stock 
portfolio for the years 1987 through 
1990 have yielded impressive re- 
sults in predicting which stocks pro- 
duce returns better or worse than 
expected." 

I The lure of networks 

Obviously, developing neural net- 
works that can accurately predict 
what was once thought to be unpre- 
dictable is a tantalizing prospect. At 
present, people are using them to 



guess the outcome of everything 
from the effects of a war on oil 
prices to the outcome of a horse 
race. It's true that a lot of these 
stories sound like hype, especially 
when they're playing to an audi- 
ence of computer professionals 
who've seen their share of flash-in- 
the-pan technologies over the 
years. Still, there is an element of 
mystery to many neural network 
applications. 

"We think there's a lot more go- 
ing on, particularly in financial cir- 
cles, than people are letting on," 
says Adaptive Solutions' Hammer- 
strom. "After all, if you had the 
inside story on the stock market, 



Neural nets give tin ears a good name 



M 



lany applications that are suited for 
neural computing are tasks at which hu- 
mans are better. But computers have an 
advantage over humans. They don't get 
tired or bored — even after several hours 
of repetitious work. With this in mind, 
engineers at cts (Matamoros, Mexico) 
made use of a neural network in their 
loudspeaker manufacturing process. 

At its plant, cts manufactures several 
million loudspeakers per year. To ensure 
the quality of the units, a final inspec- 
tion was performed by a trained opera- 
tor, skilled at identifying audio defects. 

This method had some disadvan- 
tages. An operator was required to lis- 
ten to 2,000 or 3,000 speakers in a sin- 
gle day. Since the final test is very 
subjective, over the course of a day the 
operator's fatigue level, emotional state 
and stress level affected the evaluation. 
And, because the test is sub- 
jective, pass/fail criteria varied 
from person to person. As a re- 
sult, many perfectly good 
speakers were rejected, reduc- 
ing the factory's yields. 

To solve this problem, cts 
wanted to remove the subjec- 
tivity from the testing pro- 
cess — or at least to make test- 
ing consistent from one 
production run to the next. At 
first the company used pc- 
based audio test equipment 
that was sensitive enough to 
measure distortions caused by 
speaker defects. 

But this alone wasn't 
enough. Testing also had to 



classify the units as good or bad. At 
first, statistical pass/fail limits were en- 
forced. These turned out to be very unfor- 
giving and depended too much on consis- 
tent equipment and fixture setup. In 
addition, the equipment didn't distinguish 
among the various types of defects. 

It was at that point that Rick Bono, a 
design engineer at cts, decided to try us- 
ing a neural network to classify the re- 
sults of production testing. A back-prop- 
agation neural net was established, 
consisting of 10 input nodes, 18 hidden- 
layer nodes and 4 output nodes. 

To begin with, a piece of test equip- 
ment measures the loudspeakers at dif- 
ferent frequencies. The distortion values 
at these frequencies are the inputs to 
the neural net. The four possible out- 
puts identify good speakers and three 
classes of defective units. 




To train the neural net, cts ran over 
200 units, both good and bad, through 
the process, while manually classifying 
them into one of the four output clas- 
ses. This required about 20 minutes of 
computer time on a 25-MHz 486. Once 
the network was trained, Bono inte- 
grated it into the test software using 
the NeuroShell run-time code generator 
from Ward Systems Group (Frederick, 
md). Network values were developed for 
each speaker model. 

cts has been using the neural-based 
system for a year and a half and now 
runs all its products through this test- 
ing process. Over this period of time, 
the company has seen significant im- 
provements in production yields, 
thanks to the neural net's ability to 
classify consistently. 
Bono is now considering developing a 
second-generation system 
which would incorporate new 
cases as it works, train invisibly 
and then implement the new 
network. This would require a 
hardware-based implementa- 
tion of back-propagation, says 
Bono. 

Other upgrades might in- 
clude adding more output 
nodes to the neural net. 
"We've seen different cases 
pop up — different defect cases 
that weren't included in the 
original training," says Bono. 
"They have distinct patterns, 
and the net can be trained to 
recognize them." 
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Your resource for neural hardware and software 



DEVELOPMENT TOOL 
VENDORS 

List courtesy of Martin 
Middlewood and Tom 
Schwartz. 

* Adaptive Solutions 

CNAPS: Development en- 
vironment includes CNAPS 
assembler and library of 
neural network algorithms. 
(503) 690-1236 /Circle 366 

Al Ware 

N-NET EX: User and pro- 
gram interfaces, functional 
link net architecture, asso- 
ciative recall. Supervised and 
unsupervised learning. 
(216) 421-2380 / Circle 367 

AND America 

HNet: Neural-based de- 
velopment system using dig- 
ital holography principles. 
Transputer- and pcAA/in- 
dows-based versions. 
(416) 569-0897 (Canada) 
Circle 368 

Applied Cognetics 

WinBrain: Develops back- 
propagation networks. In- 
corporates multiple trans- 
formational models. 
(212) 969-8769 / Circle 369 

* California Scientific 
Software 

Brainmaker: Basic version 
supports up to 512 input 
neurons, up to six hidden 
layers. Tutorial and eight 
sample networks. Imports 
Excel, Lotus, dBase, binary, 
and ASIC files. Print/edit 
neuron matrices. 
(800) 284-8112 / Circle 370 

EPIC Systems Group 

Neuralyst: Integrates 
neural networks with Excel 
spreadsheets. Includes 
macro library for investment 
analysis. 

(818) 564-0383 / Circle 371 

* HNC 

ExploreNet 3000: Win- 
dows-based application 
software program for devel- 
oping and implementing 
neural network solutions 
without programming. 
Database mining program 
available also. 
(619) 546-8877 / Circle 372 



Hyperlogic 

Owl Neural Network Li- 
brary: Twenty-four functions 
for accessing networks sup- 
plied as C library. Twenty 
types of neural networks. 
(619) 746-2765 / Circle 373 

Image Soft 

ExperNet: Object-ori- 
ented tool for creating Win- 
dows-based neural net- 
works and knowledge 
applications. 

(800) 245-8840 / Circle 374 

Inductive Solutions 

NNetSheet: Supports 
nine algorithms for super- 
vised and unsupervised 
training. Train network can 
be ported to a spreadsheet. 
(212) 945-0630 / Circle 375 

Mathworks 

Neural Network Toolbox: 
Includes learning rules, 
transfer functions and train- 
ing and design procedures 
for implementing neural 
networks. 

(508) 653-1415 /Circle 376 
Martingale Research 

SYSPRO: FORTRAN-baSed 

neural network simulation 
and prototyping tool. 
(214) 422-4570 / Circle 377 

Neural Computer 
Sciences 

NeuralDesk: Supports 
many algorithms. Manual 
and automatic training of 
neural networks. 
44-703-667775 (UK) 
Circle 378 

Neural Systems 

Genesis: Development 
environment for interfacing 
neural networks to applica- 
tion software. 
(604) 263-3667 (Canada) 
Circle 379 

* NeuralWare 

NeuralWorks: Neural net- 
work chip development, 
open architecture, 8-k back- 
propagation, makes net- 
work types from libraries 
and creates diagnostic tools. 
(412) 787-8222 / Circle 380 

Neurix 

MacBrain: Flexible neural 
connections, activation 
rules, 3-D graphs, interactive 



modeling, visual macro lan- 
guage. 

(617) 426-5096 / Circle 381 

* NeuroDynamX 

DynaMind: Train net- 
works on Intel's 80170NX 
etann and Intel multichip 
board. Can read and store 
network trained in emula- 
tion mode and download 
weights to chip. 
(800) 747-3531 / Circle 382 

NeuroSym 

Neural case: Supports 
four network paradigms: 
bpn, cpn, rn, and SOM. 
(713) 523-5777 / Circle 383 

Peak Software 

Autonet: Constructs net- 
works from training data 
sets consisting of input vari- 
ables and expected results. 
Networks may also be cre- 
ated from command line. 
(612) 854-0228 / Circle 384 

SAIC Artificial Neural 
Systems 

Delta ANSpec: Language 
for defining and implement- 
ing parallel distributed pro- 
cessing systems. 
(619) 546-6005 / Circle 385 

Software Bytes 

ET 2.0: Simulates text, 
graphics and Windows. 
Back-error propagation 
neural networks with Bor- 
land C/C++ source code, ET 
Graphics and Windows slide 
neural networks on equiva- 
lent VGA screens. 
(800) 521-4119 / Circle 386 

Software Frontiers 

Neural Network Toolkit: 
Development software for 
neural network applications. 
C source code included. 
(800) 475-9082 / Circle 387 

Talon Development 

Brain: Lotus 1-2-3 add-in 
for neural net development. 
(414) 962-7246 / Circle 388 

★ Ward Systems Group 

NeuroShell: Shell pro- 
gram imports ascii spread- 
sheet problem files. Example 
included. Windows version 
can build up to 128 inter- 
acting networks. Supports 
dBase. Includes run-time op- 
tion. (301) 662-7950 
Circle 389 



CHIP AND 

HARDWARE VENDORS 

★ Adaptive Solutions 

CNAPS System: A 5-billion- 
connection-per-second 
neurocomputer. The sys- 
tem has a back-propaga- 
tion learning rate of 1 bil- 
lion connection updates 
per second. 

(503) 690-1236 /Circle 390 

American NeuralLogix 

NLX420: Neural proces- 
sor slice. A digital chip de- 
signed for real-time neural 
network systems. This 20- 
MHz device contains 1 6 pro- 
cessing elements, and can 
have up to 64,000 16-bit 
synaptic inputs. 

(407) 322-5608 / Circle 391 

★ Intel 

80170NX etann: An elec- 
trically trainable analog 
neural network chip. One 
chip can perform over 2 bil- 
lion multiply-accumulate 
operations per second. 

(408) 765-9235 / Circle 392 

★ Neural 
Semiconductor 

CNU3232: Digital neural 
net chip implements a sin- 
gle-layer network of 32 in- 
puts and 32 nodes, capable 
of processing 100,000 pat- 
terns per second. 
(619) 931-7600 / Circle 393 

★ Ricoh 

RN-200: A neural net chip 
that implements a 256-syn- 
apse neural net composed 
of 16 synapses by 16 neu- 
rons. The chip is a 200,000- 
gate array built on 0.8-|im 
cmos. It has a forward pro- 
cess of 3 billion connections 
per second and a learning 
speed of 1.5 billion connec- 
tions per second. 
(408) 432-8800 / Circle 394 

★ Synaptics 

1-1000: Analog neural 
network chip designed spe- 
cifically for reading checks. 
A neural network-based im- 
age sensor reads the image 
and a neural network 
trained to recognize the 
characters on a check inter- 
prets them. 

(408) 434-0110 / Circle 395 



* Starred vendors were mentioned in this report 
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would you boast about it, or quietly 
use it to get rich?" 

Perhaps the only area where more 
mystery prevails than on Wall 
Street is in defense-related neural 
network applications. Most of the 
people involved in such projects are 
understandably reluctant to discuss 
the details of their activities, but it's 
clear that neural networks are being 
used for such things as missile guid- 
ance systems. 

"We became interested in neural 
networks about six or seven years 
ago, when other artificial intelli- 
gence solutions proved to 
be too slow for our applica- 
tions," says Dr. David 
Andes, research fellow at 
the Naval Air Warfare C en- 
ter (China Lake, ca). "As 
you can imagine, the life of 
a missile is very short — of- 
ten only a few seconds. 
The amount of room in 
which you have to store a 
guidance system and its 
power source is also pretty 
limited. After all, the pur- 
pose of the device is to 
deliver explosives, not 
electronics. We got into 
neural networks because 
biologicalbrains dothetype 
of computing that we need, 
and they do it very fast and 
in a very small area. We're 
trying to build a guidance 
system that can hit a target 
without needinghuman in- 
tervention." 

Naturally, for these applications a 
neural network must be trained to 
recognize a target in the confusion 
of battle, something that heat-seek- 
ing devices find problematic. But 
trying to give a neural network 
enough data to find targets in rap- 
idly changing battle situations is a 
daunting task. 

"Neural networks are notorious 
for picking up on things that you 
don't want them to," Andes cautions. 
"We heard about one application 
where a neural network was picking 
out the enemy from a visual field 
with perfect accuracy. Naturally, ev- 
eryone got suspicious and investi- 
gated more closely. It turned out 
that the system produced a 60-cycle 
hum whenever the picture of the bad 
guy was shown, so the network just 
incorporated that into the equa- 



tion — if hum, bad guy; if no hum, 
good guy. They took away the hum 
and the network was lost." 

I Neural network's impact 

When the breadth of neural net- 
working applications is examined, 
it's clear that if they're refined, they 
will affect our lives as much as any 
technological breakthrough of the 
twentieth century. But trying to find 
out where to separate fact from fic- 
tion is difficult, particularly when 
those who are really successful with 
neural networks are reluctant to di- 




Banks are using neural network software to verify a person's 
loan eligibility. In this application from HNC, variables such 
as an applicant's time at a fixed address, time at a particular 
job, previous loan delinquency, and savings account data are 
passed through the network for evaluation. The network 
builds a profile of the loan applicant and makes a recommen- 
dation to the financial institution. 



vulge too many details. And so the 
questions remain — can you use a 
neural network in your job, and how 
will neural networks ultimately af- 
fect your life? 

As far as jobs are concerned, 
neural networks are best suited to 
analytical tasks that prove too com- 
plex or tiring for humans to perform 
accurately. And certainly because 
neural networks are based on com- 
puter technology, they will affect the 
electronics industry if they're widely 
embraced. 

Technological applications of 
neural networks are, in fact, start- 
ing to see the light of day. Last Feb- 
ruary, for example, Intel announced 
a breakthrough in neural network- 
ing, the capability not only to iden- 
tify patterns but also to read out 
their locations. The company is us- 



ing this technology to analyze 
failures at its fabrication lines. Lo- 
cation information is reported in 
aeronautics-style polar coordinates. 
For example, a defective side of a 
6-in. wafer may be at 6 o'clock — like 
a pilot reporting an enemy's position 
in the sky. 

Neural networks could also con- 
ceivably guide placement and rout- 
ing algorithms for chip and printed 
circuit board design, in essence 
making decisions intuitively, like ex- 
perienced engineers, rather than 
through slavish adherence to algo- 
rithms. The potential is 
only limited by your imag- 
ination and by the amount 
of data that you have. 

"When we qualify poten- 
tial customers' applica- 
tions, we go by how much 
organized data we have on 
them, not on the size of 
their checkbooks," says 
hnc's Ted Crooks. "There 
are plenty of companies 
out there who have all this 
data lying around. With a 
little ingenuity we can 
turn a large database that 
used to be a nuisance into 
an asset. But without that 
data we don't even bother. 
All we'd get would be an 
aggravated customer." 

It's true that we are a 
society awash in data. 
The government and the 
banking industry alone 
probably have enough 
statistics about most of us to predict 
our voting habits, buying patterns 
and loan eligibility. 

If all of this sends an Orwellian 
chill up your spine, you're not alone. 
Although it's far too soon to predict 
whether neural networks are an- 
other headline-grabbing ai story or 
the beginning of a computer revolu- 
tion, one thing is clear. The photo- 
graphs that accompany this article 
are taken from real applications. 
Somewhere a computer might be 
grading the apples that you eat. It 
may be deciding whether to give you 
a loan or recommend you for a job 
interview. And because most of you 
reading this article are familiar with 
what computers can and can't do, 
you're either smiling right now — or 
feeling a little queasy. ■ 
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